20 Oct 2019

A (typical?) story

of a young researcher

slides: osf.io/64mbt

Incentive structures

in scholarly communication

novelty

positive results

(Nosek, Spies, and Motyl 2012)

slides: osf.io/64mbt

Degrees of freedom

in the research process

Degrees of freedom

in the research process

  • multitude of decisions to be made


Are football (soccer) referees more likely to give red cards to players with dark skin than to players with light skin?

  • 29 research teams
  • same data set
  • same results?

(Silberzahn and Uhlmann 2015)

slides: osf.io/64mbt

Degrees of freedom

in the research process

  • multitude of decisions to be made


Are football (soccer) referees more likely to give red cards to players with dark skin than to players with light skin?

  • 29 research teams
  • same data set
  • same results?

(Silberzahn and Uhlmann 2015)

slides: osf.io/64mbt

Nobody’s perfect - cognitive biases

Nobody’s perfect - cognitive biases

Nobody’s perfect - cognitive biases

confirmation bias

hindsight bias

(Munafò et al. 2017)

slides: osf.io/64mbt

QRP

Questionable Research Practices

outcome switching/ failing to report all DV

failing to report all conditions

selectively reporting studies that “worked”

(John, Loewenstein, and Prelec 2012)

selective
reporting

slides: osf.io/64mbt

QRP

Questionable Research Practices

peeking & optional stopping

excluding data (after looking at the impact)

HARKing

(John, Loewenstein, and Prelec 2012)

flexibility
in methods

slides: osf.io/64mbt

Open Science (Practices)

What is Open Knowledge?

Knowledge is open if anyone is free to access, use, modify, and share it - subject, at most, to measures that preserve provenance and openness.



Open Knowledge Foundation

slides: osf.io/64mbt

What is Open Science?

Open science is the movement to make scientific research, data and dissemination accessible to all levels of an inquiring society.



FOSTER Open Science
(Woelfle, Olliaro, and Todd 2011)

slides: osf.io/64mbt

Open Science Principles

  1. open access
  2. open data
  3. open source & materials
  4. open methodology
  5. open peer review
  6. open educational resoures (Piwowar and Vision 2013)

slides: osf.io/64mbt

 

(Fecher and Friesike 2014)

What’s in it for me?

What’s in it for me?

  • OA articles higher citation rate
      (Lewis 2018)
  • Preprint + Publication higher citation rate
      (Fraser et al. 2019)

slides: osf.io/64mbt

What’s in it for me?

  • OA articles higher citation rate
      (Lewis 2018)
  • Preprint + Publication higher citation rate
      (Fraser et al. 2019)
  • Open data higher citation rate
      (Piwowar and Vision 2013)
  • Data papers in Open Data Journals
      (incomplete list of journals)

slides: osf.io/64mbt

What’s in it for me?

  • OA articles higher citation rate
      (Lewis 2018)
  • Preprint + Publication higher citation rate
      (Fraser et al. 2019)
  • Open data higher citation rate
      (Piwowar and Vision 2013)
  • Data papers in Open Data Journals
      (incomplete list of journals)
  • Facilitate collaboration
     
  • Feedback improvement
     

slides: osf.io/64mbt

Thank you

Pictures

Fecher, Benedikt, and Sascha Friesike. 2014. “Open Science: One Term, Five Schools of Thought.” In Opening Science, edited by Sönke Bartling and Sascha Friesike, 17–47. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-00026-8_2.

Fraser, Nicholas, Fakhri Momeni, Philipp Mayr, and Isabella Peters. 2019. “The Effect of bioRxiv Preprints on Citations and Altmetrics.” Preprint. Scientific Communication; Education. https://doi.org/10.1101/673665.

John, Leslie K., George Loewenstein, and Drazen Prelec. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–32. https://doi.org/10.1177/0956797611430953.

Lewis, Colby Lil. 2018. “The Open Access Citation Advantage: Does It Exist and What Does It Mean for Libraries?” Information Technology and Libraries 37 (3): 50–65. https://doi.org/10.6017/ital.v37i3.10604.

Munafò, Marcus R., Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Du Percie Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. Ioannidis. 2017. “A Manifesto for Reproducible Science.” Nature Human Behaviour 1 (January): 0021. https://doi.org/10.1038/s41562-016-0021.

Nosek, Brian A., Jeffrey R. Spies, and Matt Motyl. 2012. “Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth over Publishability.” Perspectives on Psychological Science : A Journal of the Association for Psychological Science 7 (6): 615–31. https://doi.org/10.1177/1745691612459058.

Piwowar, Heather A., and Todd J. Vision. 2013. “Data Reuse and the Open Data Citation Advantage.” PeerJ 1 (October): e175. https://doi.org/10.7717/peerj.175.

Silberzahn, Raphael, and Eric L. Uhlmann. 2015. “Crowdsourced Research: Many Hands Make Tight Work.” Nature 526 (7572): 189–91. https://doi.org/10.1038/526189a.

Woelfle, Michael, Piero Olliaro, and Matthew H. Todd. 2011. “Open Science Is a Research Accelerator.” Nature Chemistry 3 (10): 745–48. https://doi.org/10.1038/nchem.1149.